Generating contexts based on intent description of content provider

ABSTRACT

Techniques for improving dynamic content retrieval are described. In an example, a computer system may receive, from a content provider device, an intent description associated with providing content at a network resource. The computer system may determine a goal and a predicate based on the intent description and may select a workflow from a plurality of workflows based on the goal and the predicate. The workflow may be associated with determining a context of a network document based on historical behavior data of visitors to the network resource and on a taxonomy of the network resource. The computer system may determine the context based on an execution of the workflow, the historical behavior data, and the taxonomy. The computer system may store an association between the context and the content. The association may be usable to select the content for presentation in a network document of the network resource.

BACKGROUND

Users operate computing devices to access different types of networkdocuments from network resources, such as web pages from web sites.Typically, the network access of a computing device is supported by adynamic content retrieval. In particular, content can be dynamicallyselected and retrieved from a content source and inserted in a networkdocument provided to the computing device. In this way, relevant contentcan be presented at the computing device, thereby generally improvingthe dynamic content retrieval and network resource utilization (e.g., byproviding relevant content instead of random content).

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 illustrates an example of intent construction and intentsatisfaction associated with providing content, according to anembodiment of the present disclosure;

FIG. 2 illustrates an example of a network environment for generatingcontexts based on an intent description and providing content based onthe contexts, according to an embodiment of the present disclosure;

FIG. 3 illustrates an example of interactions between computingresources for generating contexts based on an intent description,according to an embodiment of the present disclosure;

FIG. 4 illustrates an example flow for generating contexts based on anintent description, according to an embodiment of the presentdisclosure;

FIG. 5 illustrates an example of a network environment for providingcontent based on contexts, according to an embodiment of the presentdisclosure;

FIG. 6 illustrates an example of interactions between computingresources for providing content based on contexts, according to anembodiment of the present disclosure;

FIG. 7 illustrates an example flow for providing content based oncontexts, according to an embodiment of the present disclosure; and

FIG. 8 illustrates a computer architecture diagram showing an examplecomputer architecture, according to an embodiment of the presentdisclosure.

DETAILED DESCRIPTION

In the following description, various embodiments will be described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described.

Embodiments of the present disclosure are directed to, among otherthings, improving the dynamic content retrieval and network resourceutilization. In particular, contextual content targeting may beimplemented, where contexts may be defined based on an intent of acontent provider and the content selection may be performed based onsuch contexts. The context definition and the content selection may bereferred to herein as intent construction and intent satisfaction,respectively.

In an example, to generate contexts under the intent construction, acomputer system of a service provider may receive an intent descriptionfrom a computing device of a content provider. The intent descriptionmay indicate a goal and predicates (e.g., constraints) for using contentof the content provider. Based on the goal and predicates, the computersystem may select and execute a workflow to identify applicable contextsfrom historical behavior data of visitors to network resources where thecontent could be presented and based on ontologies and taxonomies ofsuch resources. Once identified, the computer system may associate thecontexts with the content of the content provider, where such contextscan be subsequently used to select the content under the intentsatisfaction.

Under the intent satisfaction, the computer system may receive a contentrequest from a network system for content to be inserted in a networkdocument requested by a computing device of a user. The computer systemmay augment the content request by adding contexts associated withnetwork document, the computing device, and the user. Some or all thesecontexts may be pre-computed under the intent construction. The computersystem may match the contexts to contexts associated with candidatecontent of different content providers. To select one of the candidatecontent, the computer system may score each of the candidate contentbased on the matched contexts, the historical behavior data, and theintent description of the corresponding content provider. The scores maybe used, among other factors, to select the content that is theninserted in the network document.

To illustrate, consider an example use case of advertisement targetingon web pages. A service provider may administer multiple web sites. Afirst web site may offer items for sale, while a second web site mayoffer a video streaming service. An advertiser may be interested inshowing an advertisement for their product line of basketball shoes onboth web sites. Accordingly, the advertiser may operate a computingdevice to access an interface to an advertisement service of the serviceprovider. At the interface, the advertiser may input their intent to“show my basketball shoes where I am likely to get purchases.” Under theintent construction, the advertisement service may determine that theadvertiser's goal includes user conversion and that a predicate includesbasketball shoes. The advertisement service may then execute a workflow,such as a machine learning (ML) algorithm, that processes historicalbehavior data of web visitors to the two web sites to identify contextsthat resulted in user conversions when exposed to advertisements relatedto shoes and/or basketball. The contexts may depend on the ontology andtaxonomy of each web site. For example, basketball shoes may have adifferent meaning under each web site (e.g., an item available for saleunder the first web site and a video about a sport event under thesecond web site) and may correspond to different categories (e.g., ashoe category under the first web site and a video genre under thesecond web site). Hence, the contexts may include different keywordsfrom visited web pages of the web sites and/or attributes of keywords(e.g., “basketball shoes”, “sports shoes”, “athletic wear”, “comfortableshoes”, “Brand ‘ABC’ shoes”, “Brand ‘XYZ’ shoes”, and other contexts forthe first web site, and “basketball game”, “sport event”, “basketballplayer ‘John Doe’ documentary”, and other contexts for the second website). The advertisement service may store associations between suchcontexts and the advertisement about the advertiser's basketball shoes.

Upon a web visitor to either web sites, the advertisement service mayreceive an advertisement request identifying, for example, the visitor'srequested web page (e.g., a particular item detail page from the firstweb site or a particular movie page from the second web site). Theadvertisement service may retrieve pre-computed contexts associated withthe requested web page and may on-the-fly compute visitor contexts(e.g., visitor's location and/or user segment to which the visitorbelongs). Such context may be added to the advertisement request,thereby augmenting the request to include request contexts. Further, theadvertisement service may match the request contexts with pre-computedcontexts that are associated with the advertiser's advertisement aboutbasketball shoes and with other advertisements. Accordingly, variouscandidate advertisements may be identified as being relevant forinsertion in the requested web page. The advertisement service may use aML algorithm to score these advertisements, where each is scored byinputting the contexts and the corresponding intent description to theML algorithm. The scores may be used among other factors (e.g.,placement rate, advertisement campaign budget, advertisement bid amount,visitor fatigue, and the like) to select the basketball shoesadvertisement. This advertisement may then be inserted in the requestedweb page when presented to the web visitor.

Embodiments of the present disclosure provide many technical advantagesover existing dynamic content retrieval systems, as further described inconnection with the next figures. For example, the interface to adynamic content retrieval system may be significantly improved, where itmay be sufficient to a content provider to define their intent usingnatural language. Further, given an extensive knowledge set aboutbehavior data, the intent may be translated into a set of contexts byusing, for instance, artificial intelligence (AI). Accordingly, thecontexts may more accurately define contextual signals that otherwise,would not have been defined by the advertiser. Such contexts can bemonitored and used to select and retrieve more relevant content forinsertion, thereby improving the performance of the content retrievalsystem. In turn, because more relevant content is selected, theutilization of the underlying computing resources may be optimized,whereby these resources may no longer need to process unrelated orrandom content.

In the interest of clarity of explanation, various embodiments aredescribed herein in connection with selecting and inserting content in aweb page. However, the embodiments are not limited as such and apply toany other type of network documents of a network resource. For example,a computing device of a user may request access to a network documentvia an application programming interface (API) specific to the networkresource. This access request may trigger a content request that maythen be augmented with contexts. These contexts may then be used toselect particular content that may be inserted and presented in thenetwork document.

In embodiments, depending on the type of user information obtainedand/or analyzed, appropriate measures are taken to ensure protection ofthe collected data, to empower the users to determine what data can beused for the purposes corresponding to the intent construction andintent satisfaction, as well as empower the users to indicate whichportions of the collected data or user information should be deletedand/or no longer obtained by the service provider computers implementingthe intent construction and intent satisfaction described herein.

FIG. 1 illustrates an example of intent construction and intentsatisfaction associated with providing content, according to anembodiment of the present disclosure. As illustrated a service providermay operate a computer system 102 to receive an intent description froma computing device 104 of a content provider (referred to herein ascontent provider device for brevity). The computer system 102 maytranslate the intent description into a set of contexts and mayassociate this set with content of the content provider. A web visitormay operate a computing device 106 (referred to herein as visitor devicefor brevity) to access a web page of a web site. The computer system 102may then use the contexts to insert the content of the content providerinto the web page presented at the visitor device 106.

In an example, the computer system 102 may host a context-contentservice that provides 110 an interface to the content provider device104. Generally, the interface may include a web interface or an API thatallows the content provider to define an intent for using their content.The interface may provide additional functionality such as, informingthe content provider about a potential performance of the content giventhe intent, allowing a selection of identified contexts, and/or allowinga selection of specific web sites for which the content would beavailable.

The interface may support one or more interaction modalities, such as anaudible modality (e.g., where the content provider may interact with theadvertisement service by uttering words to a voice-based device), agraphical modality (e.g., where the content provider may interact withthe advertisement service via a graphical user interface (GUI) 112,and/or a combination of audible and graphical modalities. As illustratedin FIG. 1, the interface may provide a GUI 112 that is presented at thecontent provider device 104. The GUI 112 may include multiple fieldscorresponding to the supported functionalities including, for instance,an input field 114, web site fields 116A-116K (generally referred toherein as a web site field), and web site selection fields 118A-118K(generally referred to herein as a web site selection field). The inputfield 114 may receive an intent description of the content provider.This intent description may be entered as a natural language (NL) inputat the input field 114. Generally, the intent description may define agoal of the content provider associated with presenting the content anda set of predicates, such as constraints and/or clauses that depend on aweb site's taxonomy, for the presentation. The web site field mayidentify a web site by, for instance, presenting a domain name or auniversal record locator (URL) of the web site. The web site selectionfield may allow a selection by the content provider of a correspondingweb site. Once a web site is selected, the context-content service maygenerate contexts for that web site based on the web site's ontology andtaxonomy, among other factors. Of course other arrangements of the GUI's112 fields may be possible. For instance, rather than presenting a website field and a web site selection field, the GUI 112 may includeanother input field for receiving an identifier of the web site.Alternatively, the input field 114 may also be used to input theidentifier, along with the intent description.

Upon receiving the intent description and, as applicable, the selectionof web site(s) via the interface, the computer system 102 may define 120contexts to construct the intent of the content provider. In an example,the context-content service of the computer system 102 may determine thegoal and predicate(s) from the intent description and may select aworkflow. The workflow may include an executable analysis process thatevaluates, based on historical behavior data, web pages and traits ofvisitors and visitor devices to generate contexts. The historicalbehavior data may include visitor journeys through the selected website(s) (e.g., visited web pages), visitor interactions and results(e.g., clickstreams and conversions), visitor traits (e.g., usersegments), and visitor device traits (e.g., locations, device types,browser types, operating system types, etc.). Each of the contexts mayrepresent contextual information determined for a particular intentdescription from historical behavior data and applicable to a web site'sontology and taxonomy. For instance, a context associated with a webpage may include a keyword found in the body of the web page (e.g., anidentifier of a particular item in an item detail page, such as “shoesof brand ‘ABC’”) or an attribute of a plurality of keywords (e.g., anitem category for different items or a value of a property of an item,such as “sports shoes” or “red shoes”). Generally, the attribute maycorrespond to a category defined based on the taxonomy of the web siteor to a property value available given the taxonomy. Similarly, acontext associated with a visitor may identify a visitor trait, whereasa context associated with a visitor device may identify a visitor devicetrait.

In addition, the context-content service may measure a performanceassociated with using the contexts to select and insert the content.This performance may represent a performance measurement of the contentprovider's intent. Various performance measurement types may bepossible, including an uncontended performance measurement (e.g., thetotal audience size that can be exposed to the content, without othercompeting content) and a contended performance measurement (e.g., thepercentage of the total audience size when other competing content areconsidered). Further, the context-content service may derive similarperformance measures for web sites that were not selected by the contentprovider by deriving the related contexts. Depending on the performance(e.g., if the performance of an unselected web site is better than thatof a selected web site), a recommendation may be made for selecting sucha web site.

The interface of the context-content service may support these and otherfunctionalities by providing the relevant data to and facilitatinginteractions of the content provider. For example and as illustrated,the GUI 112 may be refreshed to present many fields including contextfields 122A-122M (generally referred to herein as a context field),context selection fields 124A-124M (generally referred to herein as acontext selection field), a performance measurement field 126, a website recommendation field 128, and a content upload field 129. A contextfield may identify a context derived from the execution of the workflow.A context selection field may allow a selection of the correspondingcontext field by the content provider, where upon the selection, thecontext-content service would associate the context with the content ofthe content provider. These two types of fields may be optional and neednot presented to the content provider. In this way, the content providermay only need to define their intent without having to select contexts.The performance measurement field 126 may present the performancemeasurement for the intent. The web site recommendation field 128 maypresent a recommendation to select one or more web site(s) that were notselected by the content provider. The content upload field 129 maypresent an option for the content provider to accept the intentdescription and resulting contexts (selected, as applicable) and uploadthe content with which the intent description and contexts should beassociated. The content upload may include receiving, by the computersystem 102, the actual content from the relevant content source orreceiving a network address location (e.g., a URL) where this contentmay be stored.

Upon the content upload, the computer system 102 may generate and storeassociations between the contexts and the content. The associations maybe used to support contextual content targeting, including verticalsearched by content servers. As illustrated, the computer system 102 maydetermine 130 contexts for intent satisfaction upon the visitor device's106 access to a web page 132. For example, the context-content servicemay augment a content request with contexts associated with the web page132, the visitor, and the visitor device 106. These contexts are thenmatched with context-content associations to identify a set of candidatecontent. The context-content may generate a relevancy score for eachcandidate content in the set given the contexts and the correspondingintent description. The scores may be used among other factors to selectparticular content 134 from the set. Hence, upon presentation at thevisitor device 106, the web page 132 may include content 136 from theweb server hosting the web page 132 (e.g., this content may be the bodyof the web page), and the particular content 134 of the contentprovider.

To illustrate, consider the example use case of advertisement targetingon web pages of a web site that offers items for sale. The web site maybe hosted on the computer system 102. An advertiser may operate thecontent provider device 104 to access the GUI 112 and may input anintent description of “show my basketball shoes to as many users thatvisit web pages offering shoes for less than a $100.” The computersystem 102 may then derive different contexts from that intentdescription and may surface a resulting performance measurement on theGUI 112. Given this measurement, the advertiser may decide to refinetheir intent description by inputting “show my basketball shoes on anyweb page where I am likely to get purchases” at the GUI 112. One ofthese contexts may be “biography book of player ‘John Doe’.” Thecomputer system 102 may derive again contexts applicable to this intentand refresh the GUI 112. Satisfied with the expected performance, theadvertiser may upload their advertisement about their basketball shoes.The computer system 102 may store associations between the contexts ofthe refined intent description and the advertisements. Upon a visitoraccess to a detail page of the web site offering the biography book ofplayer ‘John Doe’, the computer system 102 may match this context withthe context associated with the advertisement and may score theadvertisement. Given a comparison of the score with those of othercandidate advertisements, the computer system 102 may select and insertthe basketball shoes advertisement in the detail page about thebiography book of player ‘John Doe’.

In various examples, the computer system 102 hosting the context-contentservice may also host the selectable web site(s). In this way, behaviordata may be collected by the computer system 102 and used for the intentconstruction. However, the embodiments may not be limited as such. Forexample, the embodiments similarly apply to a computer environment,where the computer system 102 may host the context-content service butnot a web server that hosts a selectable web site. In this case, acollection server of the computer system 102 may receive web referralsand redirections from the web server to then derive the behavior data ormay import the behavior data from a third party. In both cases, thecomputer system 102 may similarly use the behavior data to derive thecontexts.

By using the GUI 112 as illustrated in FIG. 1 (or by relying on thefunctionalities of the described interface), many technical advantagesover existing content retrieval system (including content targetingsystems) may become possible. For instance, and referring to theadvertisement use case, a typical contextual targeting advertisementsystem necessitates the advertiser to identify relevant web pages oneach site where their advertisement could be shown. More specifically,the advertiser may need to understand the ontology of each web site, andidentify the relevant keywords where their advertisement is to be shown.However, it is burdensome for the advertiser to identify the mostrelevant keywords across different web sites and keep them up to date.This is problematic for many reasons. For instance, advertising ondifferent web sites may need advertisers to understand differentontologies and taxonomies. Even if the advertiser is able to ingest andunderstand such ontologies and taxonomies, the advertiser may still needto identify the keywords that are relevant to their product. Dependingon the number of keywords and how the advertiser defines relevancy, thiscan be a daunting challenge. Further, different advertisers havedifferent intents that are satisfied by different keyword sets. Inparticular, an advertiser whose intent is to increase their conversionrate is looking to show their product on contextually similar pages thatare likely to drive sales. In comparison, an advertiser whose intent isreach or product awareness, their intent may be satisfied with a broaderrange of products that are relevant to the advertiser's brand regardlessof direct sales numbers. The number of advertiser intents increases withthe number of possible website ontologies and taxonomies, making afurther challenge to identify the keywords that satisfy any givenadvertiser intent. In addition, for some advertiser intents, the set ofrelevant keywords may change over time. In order to keep this set fresh,an advertiser should constantly scan for new keywords and revisitpreviously selected keywords to see if they are still relevant. Incontrast, embodiments herein when implemented for advertisementtargeting may provide a contextual targeting solution that acceptsintents from advertisers for different ontologies and taxonomies andautomatically satisfies those intents with relevant contexts, includingkeywords and other contextual information. This may shift the burden ofunderstanding different advertising domains and identifying relevantcontexts from the individual advertisers to the service provider'scomputer system 102. An advertiser may need to simply input their intentat the GUI 112. By relying on an extensive body of knowledge aboutbehavior data, the computer system 102 may derive the most relevantcontexts to use for the advertisements.

FIG. 2 illustrates an example of a network environment for generatingcontexts based on an intent description and providing content based onthe contexts, according to an embodiment of the present disclosure. Asillustrated, the network environment may include a computer system 200,a service provider device 260, a content provider device 270, a webserver 280, and a visitor device 290. The computer system 200 may besimilar to the computer system 102 of FIG. 1 and may host acontext-content service by distributing the functionalities of suchservice across the different servers. A service provider may operate theservice provider device 260 to manage some or all of the functionalitiesof the computer system 200. For example, the service provider may definevarious workflows that the computer system 200 may use to derivecontexts. A content provider may operate the content provider device 270to define their intent and upload content. The computer system 200 mayselect one of the workflows to then derive and store contexts for thecontent provider's intent. A visitor may operate the visitor device 290to request a web page 282 from the web server 280. The web server 280may, but need not be, part of the computer system 200. The web server280 may send a content request 284 to the computer system 200, where thecontent request 284 may trigger a selection of the content provider'scontent 252 based on contexts. The content 252 may then be inserted inthe web page 282 and presented at the visitor device 290.

As illustrated, the computer system 200 may include a plurality ofcomputing resources. Each of such resources may represent computinghardware or a computing service implemented on computing hardware. Thecomputer resources may include a context discovery server 210, acollection server 220, a data store 230, a context augmentation server240, and a content server 250. Interactions between these computingresources are further illustrated in the next figure.

In an example, the context discovery server 210 may provide an interfaceto the service provider device 260 (referred to herein as anadministrator interface) and an interface to the content provider device270 (referred to herein as an intent interface). The service providermay input a workflow description 262 at the administrator interface.This workflow description 262 may include definitions of a workflow(e.g., process steps), type of behavior data that should be analyzed(e.g., user conversion data, user exposure data, etc.), type of analysis(e.g., statistical analysis, ML algorithm, data type, data sources,input types, output types, etc.), and criteria for selecting theworkflow (e.g., goals and/or predicates that can be matched to theintent description, specificity level that can be matched to aspecificity of the intent description, etc.). The context discoveryserver 210 may accordingly generate and store the workflow in the datastore 230, along with the criteria (e.g., such as in metadata of theworkflow or as a file).

The content provider may input an intent description 272 at the intentinterface and use other functionalities of this interface, as describedherein above in connection with the GUI 112 of FIG. 1. The contextdiscovery server 210 may accordingly derive and store the contexts inthe data store 230. To do so, the context discovery server 210 may relyon the workflows stored in the data store 230 and on behavior datacollected by the collection server 220.

The data store 230 may also store associations between the contextsderived by the context discovery server 210 and content of contentproviders. Further, the data store 230 may store various pre-computedcontexts for different web pages 282 and web sites, where these contextsmay be derived by using one or more workflows.

The context augmentation server 240 may receive the content request 284from the web server 280. In response, the context augmentation server240 may query the data store 230 to identify relevant pre-computedcontexts and may augment the content request by adding such contextsthereto.

The content server 250 may receive the augmented content request and mayuse its cache or query the data store for matches between the contextsincluded in the augmented content request 284 and the contextsassociated with the content of the content providers. Content that havematching contexts may be selected and added to a set of candidatecontent. The content server 250 may then score each candidate contentand select particular content from the set based on the scores. Theparticular content may be provided as the content 252 to the web server280.

FIG. 3 illustrates an example of interactions between computingresources for generating contexts based on an intent description,according to an embodiment of the present disclosure. In an example, theinteractions may include calls, such as API calls to exchange data,execute processes, and generate contexts. Some or all of the computingresources may be managed by a service provider and may be the same asthose described herein above in connection with FIG. 2. In particular,the computing resources may include a context discovery server 330, adata store 340, and a collection server 350.

As illustrated, a service provider device 310 of the service providermay send a workflow descriptions 312 to the context discovery server330. For example, the service provider may input definitions andselection criteria of different workflows at an administrator interfacepresented at the service provider device 310. The context discoveryserver 330 may receive this information and, in response, may generateand transmit workflows 332 and the selection criteria to the data store340 for storage thereat.

A content provider may operate a content provider device 320 and access,via an intent interface, the context discovery server 330. The accessmay include sending an intent description 322 to the context discoveryserver 330 via the intent interface. In response, the context discoveryserver 330 may generate and send a workflow query 334 to the data store340. This workflow query 334 may include different types of information.In an example, the context discovery server 330 may derive goal(s) andpredicate(s) from the intent description 322 and may send thisinformation in the workflow query 334. In another example, the contextdiscovery server 330 may derive a specificity level of the intentdescription 322 from the goal(s) and predicate(s) and may include thespecificity level in the workflow query 334. In yet another example, thecontext discovery server 330 may send the intent description 322 or thegoal(s) and predicate(s) in the workflow query 334 and a query engine ofthe data store 340 may derive the goal(s) and predicate(s) from theintent description 322 or the specificity level from the goal(s) andpredicate(s). In these various examples, the data store 340 may return aworkflow 342 to the context discovery server 330 based on theinformation received in the workflow query 334. Generally, the queryengine of the data store 340 may select the workflow 342 from the storedworkflows 332 based on the selection criteria by, for instance, matchinggoals and predicates and/or matching specificity levels. Returning theworkflow 342 may include sending an identifier of the workflow 342 tothe context discovery server 330.

The context discovery server 330 may identify, from the workflow 342 andthe intent description 322, a particular type of behavior data thatshould be analyzed (e.g., user conversion data, user exposure data, datacollected for a particular web site(s), etc.) and may send a behaviordata query 336 to the collection server 350. The behavior data query 336may identify the particular type of the behavior data. In response, thecollection server 350 may return the applicable behavior data 352 to thecontext discovery server 330. Alternatively or additionally, thebehavior data may also be stored at the data store rather than thecollection server 350. In such a case, the behavior data query 336 canbe sent to the data store 340 instead.

Next, the context discovery server 330 may execute the workflow 342 andanalyze the behavior data 352 accordingly. For instance, if the workflow342 invokes a particular ML algorithm, the context discovery server 330may input the behavior data 352 (or any feature map that the contextdiscovery server 330 may generate from this behavior data 352) to the MLalgorithm. The output 338 of the execution of the workflow 342 mayinclude contexts and, optionally, performance measurements andrecommendations. The context discovery server 330 may transmit theoutput 338 to content provider device 320 for presentation at the intentinterface.

The content provider may operate the content provider device 320 to senda confirmation 324 and content 326 to the context discovery server 330.The confirmation 324 may indicate that the content provider would liketo use the contexts (or a subset thereof that may be selected via theintent interface) for subsequent selection and insertion of the content326 in web pages. Of course, if any changes to the intent description322 are made, the context discovery server 330 may re-generate thecontexts and provide an update to the output 338. Sending the content326 from the content provider device 320 to the context discover server330 may include transmitting the content or an identifier thereof, suchas an address of a network location (e.g., URL) where the content may bestored.

The context discovery server 330 may generate context associations 344based on the confirmation 324 and the content 326. Further, the contextdiscovery server 330 may transmit the context associations 344 and thecontent 326 to the data store 340 for storage thereat. Generally, acontext association 344 may include an association between a contextfrom the output 338 and the content 326. Various types of the contentassociations may be possible. One example type may associate the contextwith the content 326 and another example type may have the reverseassociation (e.g., of the content 326 with the context). These two typescan be used for different purposes. For instance, the contextassociation 344 may include a first key-value pair, where the key andthe value of this pair include the context and the identifier of thecontent 326, respectively. The context association 344 may also includea second key-value pair, where the key and the value of this pairinclude the identifier of the content 326 and context, respectively. Thefirst pair may be used to match the context (e.g., the key) with adetected context associated with a visitor device access to a web pageand, accordingly identify the content 326. The second pair may be usedto return query results about content to the content provider device320. In particular, the content provider may submit a query about thecontexts associated with one of its particular content. The content inthe second key-value pair may be matched to the particular content tothen return a result about the context. In this way, the contentprovider may quickly identify the associated contexts and/or the totalnumber thereof.

FIG. 4 illustrates an example flow for generating contexts based on anintent description, according to an embodiment of the presentdisclosure. Instructions for performing the operations of the flow canbe stored as computer-readable instructions on a non-transitorycomputer-readable medium of a computer system, such as the computersystems described herein above in connection with FIGS. 1-3. As stored,the instructions represent programmable modules that include code ordata executable by a processor(s) of the computer system. The executionof such instructions configures the computer system to perform thespecific operations shown in the figures and described herein. Eachprogrammable module in combination with the processor(s) represents ameans for performing a respective operation(s). While the operations areillustrated in a particular order, it should be understood that noparticular order is necessary and that one or more operations may beomitted, skipped, and/or reordered. In the interest of clarity ofexplanation, the flow is illustrated in connection with generating acontext associated with a web page of a web site. The flow similarlyapplies to generating multiple contexts associated with a same web page,different web pages, same web site, and/or different web sites.

As illustrated, the example flow may start at operation 402, where thecomputer system may provide an interface (e.g., an administratorinterface) to a service provider device to define workflows. In anexample, the interface may include a GUI that is presented at theservice provider device. The computer system may drive the GUI.

At operation 404, the computer system may receive a workflow descriptionfrom the service provider device based on the interface. In an example,a service provider may input process definitions, analysis type,behavior data type, and selection criteria in different fields of theGUI. The inputted information may be transmitted from the serviceprovider device to the computer system.

At operation 406, the computer system may generate and store a workflowbased on the workflow description. In an example, the computer systemmay generate a set of executable instructions that define the processsteps, the applicable analysis type and behavior data type that shouldbe analyzed under each step, network addresses of the relevant processesand relevant behavior data, and metadata for the selection criteria. Thegenerated workflow may be stored in a data store accessible to thecomputer system.

At operation 408, the computer system may provide an interface (e.g., anintent interface) to a content provider device to define intent. In anexample, the interface may include a GUI that is presented at thecontent provider device, similar to the GUI 112 of FIG. 1. The computersystem may drive the GUI.

At operation 410, the computer system may receive an intent descriptionfrom the content provider device based on the interface. In an example,a content provider may input, in one or more input fields of the GUI, adescription of an intent for presenting content at the web site. Thisintent description may follow a natural language format. The inputtedintent description may be transmitted from the content provider deviceto the computer system.

At operation 412, the computer system may determine a goal(s) and apredicate(s) based on the intent description. In an example, the goal(s)and predicate(s) are determined by using query understanding to identifya query structure and natural language processing to identify thegoal(s) and predicate(s) given the structure. Further, the predicate(s)may represent a constraint or a query clause that depends on theontology and taxonomy of the web site associated with the intentdescription. In this case, the natural language processing may alsoaccount for such ontologies and taxonomies.

At operation 414, the computer system may select a workflow from aplurality of workflows based on a number of factors. In an example, thedifferent workflows may be available from the data store. The factorsmay include the intent description, the goal(s) and predicate(s), andthe specificity level. The computer system may match one or more ofthese factors with the selection criteria of a workflow to select theworkflow. In an illustration, the selection criteria may include anintent description. In this illustration, the computer system may matchthe intent description received at operation 410 with the selectioncriteria and select the workflow. In another illustration, the selectioncriteria may include a set of goals and a set of predicates. In thisillustration, the computer system match the goal(s) and predicate(s)determined at operation 412 with the selection criteria to select theworkflow. In yet another illustration, the selection criteria mayinclude a query specificity level. Accordingly, the computer system mayderive a specificity level of the intent description based on thegoal(s) and predicate(s) and may match this specificity level with theselection criteria. Different techniques for determining match may bepossible. In an example, an AI model, such as a ML algorithm, may betrained to perform the match. Once trained, the input to the AI modelmay include one or more of the factors, and the output may identify theworkflow.

In an example, the AI model is used for selecting the workflow based onthe intent description (or any of the other factors). In particular, theintent description (or the other factors) may be an input to the AImodel. The AI model may be trained to determine candidate web pages ofthe web site based on the intent description (or the other factors) andto identify candidate keywords from the candidate web pages. Each ofsuch keywords may correspond to, for an instance, an item that a webpage may describe (e.g., the item of an item detail page). The AI modelmay also be trained to compute a similarity measurement between thecandidate keywords. This measurement may indicate how similar ordissimilar the candidate keywords (or the items) may be. One type of asimilarity measurement may relate to attributes of the candidatekeywords. If the candidate keywords share a common set of attributes(e.g., these keywords have similar attributes), the similaritymeasurement may indicate that a similarity exists between theattributes. In this case, the selected workflow may include a keywordidentification workflow. Specifically, the keyword identificationworkflow may be defined and executed to, for instance, identifykeywords. Otherwise, the similarity measurement may indicate that adissimilarity exists between the attributes. In this case, the selectedworkflow may include an attribute identification workflow. Specifically,the attribute identification workflow may be defined and executed to,for instance, identify attribute(s) of one or more keywords.

To illustrate, and referring to detail pages describing items, thecandidate keywords may be identifiers of the items (e.g., specificproduct numbers). These items may belong to a number of item categories(e.g., “running shoes,” “sportswear,” “biography books,” etc. or othercategories as defined based on the web site's taxonomy) and theattributes may correspond to the item categories. If the number of theitem categories is small, the similarity measurements may indicate thesimilarity and, accordingly, the attribute identification workflow isselected. However, if the number of the item categories is large or ifthe item categories are unrelated to each other (e.g., not beingsub-categories of a same category, depending on the web site'staxonomy), the similarity measurements may indicate the dissimilarityand, accordingly, the keyword identification workflow is selected.

Different techniques for deriving the specificity level may be possible.In an example, the specificity level may depend on the number ofpredicates. Generally, the larger the number of the predicates, the morespecific the intent description (e.g., the higher the specificity level)may be. In another example, the computer system may determine a numberof expected contexts (e.g., expected keywords) based on the goal(s) andpredicate(s). This number may represent the potential size of the set ofcontexts that may be subsequently derived. The size of this set may beestimated by inputting the goal(s) and predicate(s) to a predictionmodel (e.g., a ML algorithm) trained based on historical intentdescriptions. If the number is smaller than a threshold indicating thatthe set size may be small, the computer system may determine that theintent description has a high specificity level. On the other hand, ifthe number is larger than a threshold indicating that the set size maybe too large, the computer system may determine that the intentdescription has a low specificity level. High and low specificity levelsmay be matched to different workflows. In particular, for a highspecificity level intent description, the selected workflow may includethe keyword identification workflow. In comparison, for a lowspecificity level intent description, the selected workflow may includethe attribute identification workflow.

At operation 416, the computer system may receive historical behaviordata. In an example, the computer system may determine the type of thehistorical behavior data based on the workflow and may retrieve theapplicable behavior data from the data store or a collection server. Thehistorical behavior data may include data about visited web pages of theweb site, outcomes of the access to the web pages (e.g., exposure,conversion, etc.) visitors to these web pages, and visitor devices.

At operation 418, the computer system may determine a context based onan execution of the workflow, the historical behavior data, and thetaxonomy and ontology of the relevant web site. In an example, theexecution of the workflow may initiate an analysis process (e.g., a MLalgorithm) defined by the workflow. The historical behavior data, andthe taxonomy and ontology, along with the goal(s) and/or predicate(s)from the intent description may be input to the analysis process. theoutput may be the context, where this context may be associated with aparticular web page of the web site.

To illustrate, consider two examples. In a first example, thespecificity level is high and the keyword identification workflow isselected and executed. In this example, the intent description is “Iwish to show an ad for my product ‘ABC’ on detail pages that are forBrand XYZ's athletic shoes and that likely results in conversion.” Thecomputer system may determine that the goal is user conversion andidentify multiple predicates: “product=ABC,” “Brand=XYZ,” “productcategory=athletic shoes.” The computer system may then retrievehistorical behavior data for only visitors that have visited the detailspages of Brand XYZ's athletic shoes and that have made a shoe purchase.This data may identify specific detail pages. Common keywords acrossthese pages and relevant to the brand and/or product category may beidentified based on, a statistical analysis (e.g., the top five percentor some other percentage keywords that are common) or by using a MLalgorithm. The keywords may be subject to a set of filters such thatsome of the keywords may be filtered based on a blacklist, a similarityto keywords found on the product ABC's detail page, insufficient trafficassociated with them, or have a low relevancy score (where this scoremay be generated by an artificial intelligence (AI) model as furtherdescribed in connection with the next figures). The computer system mayalso generate a relevancy score for each of the remaining keywords andmay rank these keywords. These keywords and their ranking may be storedin the data store.

In a second example, the specificity level is low and the attributeidentification workflow is selected and executed. In this example, theintent description is “I wish to show an ad for my product ‘ABC’ toincrease my product recognition” The computer system may determine thatthe goal is product reach and identify the predicate: “product=ABC”. Thecomputer system may then determine various values for attributesassociated with the predicate (e.g., shoes, athletic shoes, basketballshoes, red shoes, etc.). Optionally, the computer system may presentthese attribute values at the GUI to allow a selection of a subset ofthe attribute values by the content provider. Once the set of attributevalues are finalized (e.g., based on or absent of a selection of thecontent provider), the computer system may retrieve historical behaviordata for visitors that have visited the website and were exposed to suchattributes (e.g., visited any detail page that presented informationabout shoes, athletic shoes, basketball shoes, and red shoes dependingon the attribute set). This data may identify specific details pages.Common contexts (e.g., attributes and their values, where each attributemay correspond to one or more keywords) across these pages may beidentified based on a statistical analysis (e.g., the top five percentor some other percentage contexts that are common) or by using a MLalgorithm. Some of the contexts may be filtered our based on a set offilters. The computer system may also generate a relevancy score foreach of the remaining contexts and may rank these contexts. Thesecontexts and their ranking may be stored in the data store.

At operation 420, the computer system may estimate a performancemeasurement based on the context determined at operation 418. In anexample, the performance measurement may include uncontended andcontended performances for using the context. These performances may bedetermined from the behavior data. For instance, the computer system mayidentify, from the behavior data, a set of the web pages associated withthe context (e.g., the particular web pages that include the keyword, orthe particular web pages that include one or more keywords having theattribute) and outcomes of the visitor interactions with these web pages(e.g., converted, exposed, etc.). Based on the intent description'sgoal(s), the computer system may determine a subset of the set, wherethe subset corresponds to the particular web pages having an outcomethat meets the goal(s). by comparing the size of the subset to the sizeof the set, the computer system may determine the uncontendedperformance. By further assuming a particular distribution functiongiven potential contentions with content of other content providers, thecomputer system may determine the contended performance from theuncontended performance.

At operation 422, the computer system may provide the performancemeasurement to the content provider device. In an example, the computersystem may send the performance measurement to the content providerdevice with instructions to present the measurement in a performancemeasurement field of the GUI.

At operation 424, the computer system may estimate a performancemeasurement for a different web site. In an example, this other web sitemay not have been selected or identified by the content provider.Instead, the computer system may initiate a recommendation workflow todetermine whether to recommend that the content should also beassociated with and become presentable at the other web site. Therecommendation workflow may repeat operations 414-420, by generatingcontexts specific to the other web site based on the content provider'sintent description and the other web site's ontology and taxonomy. Thecomputer system may then compare the performance measure of this otherweb site with the first performance measurement of the web site selectedor identified by the content provider and/or with a predefinedperformance threshold. If the first performance measurement or theperformance threshold is exceeded, the computer system may generate therecommendation to associate the content with the other web site.

At operation 426, the computer system may provide the recommendationabout a different web site to the content provider device. In anexample, the computer system may send the recommendation to the contentprovider device with instructions to present the recommendation in arecommendation field of the GUI. As presented, the recommendation mayidentify the other web site, indicate that an association therewith isrecommendation, and present the related performance measurement.

At operation 428, the computer system may provide the context (asdetermined at operation 418) to the content provider device. In anexample, the computer system may send the context to the contentprovider device with instructions to present the context in a contextfield and a selectable option in a context selection field of the GUI.As presented, the GUI may allow the content provider to view and selectthe context for use in association with its content on the web site.

At operation 430, the computer system determines whether the intentdescription has changed. In an example, the computer system may havereceived an updated description of the content provider's intent basedon the presentation of the context, the performance measurement, and/orother web site recommendation. If so, operation 412 may follow operation430, such that the computer system may loop back to the operation fordetermining an updated context. Otherwise, operation 432 may followoperation 430.

At operation 432, the computer system may receive a selection of thecontext from the content provider device. In an example, the contentprovider may select the context via the context selection field of theGUI. The computer system may receive data indicating this selection fromthe content provider device.

At operation 434, the computer system may generate an associationbetween the context and the content. In an example, the computer systemmay also receive the content (or an identifier of the content) from thecontent provider device. Upon receiving the selection of the context,the computer system may generate and store first and second key-valuepairs, where the context and the identifier of the content (as stored bythe computer system or as available from a remote content source) arethe key and value in a first pair, respectively, and the value and thekey in a second pair, respectively.

At operation 436, the computer system may store the association in thedata store. In an example, the association may be usable to select thecontent for presentation in a web page of the web site when access tothe web page is requested by a visitor device. The selection andinsertion of the content is further illustrated in connection with thenext figures. Generally, the computer system may receive a contextrequest from the web server. Based on context matching and contentselection, the computer system may respond with instructions identifyingthe content and instructing the web server to insert the content in theweb page.

Various operations of the example flow 400 may be repeated based on atrigger event. An example trigger event may be a change to the intentdescription. Another trigger event may be independent of any change tothe intent description or any request from the content provider.Instead, this trigger event may be an elapse of a time interval (e.g.,such that the context generation and association with the content can berepeated periodically), or a change to the taxonomy of the web site orto the web pages. For instance, the context may be generated andassociated with the content at a first point in time. At a second andsubsequent point in time, the intent description may remain the same.However, the applicable workflow may be automatically re-executed at thesecond point in time based on updated historical behavior data (e.g.,including historical data that became available after the first point intime), on a change to the taxonomy of the network resource, an additionof web pages, and/or removal of web pages.

In addition to associating the content with the context (referred to inthis paragraph as a first context for clarity), the computer system mayfurther associate the content with a second context based on the secondcontext being an expected context given the first context. Inparticular, when a visitor device requests access to a web page of theweb site, a context associate with this request may be determined. Incertain situation, this context may be the first context. In thesesituations, the second context may be a next context expected to bedetected upon a next access request from the visitor device. Toillustrate, the first context may be a particular keyword (e.g., “shoes‘LMN’”) and is associated with the content (e.g., advertisement forproduct ‘ABC’). Historical behavior data may indicate that upondetecting this keyword, the next context likely to be detected isanother keyword (e.g., “shirt OPQ’”). In this case, the second contextmay be set as the other keyword and may be associated with the samecontent based on the second content being an expected context of thefirst context. In other words, the computer system may expand thecontext-content associations by generating new associations with thecontent based on expected contexts given the already associatedcontexts.

To do so, the computer system may analyze the historical behavior data(e.g., based on a prediction model that uses statistical analysis,artificial intelligence, or the like) to identify one or more expectedcontexts for each context determined under the flow of FIG. 4. Forexample, a first context and a second context may be identified fromhistorical behavior data. This data may include visitor journeys (e.g.,sequences of visited web pages). The second context may be set as anexpected context when the visitor journeys indicate that the secondcontext is likely or frequently (e.g., with a likelihood or frequencylarger than a predefined threshold) observed as the next context afterobserving the first context. For instance, if the first context is akeyword found in a first web page and the second context is a keywordfound in second and third web pages, the second context may be set anexpected context when the visitor journeys indicate that visitorsfrequently visit the second or third web page after visiting the firstweb page.

Upon generating a first association of a first context and content, andupon determining a second content that is expected given the firstcontext, the computer system may also generate a second association ofthe second context with the content. In this way, when the secondcontext detected upon an access request of a visitor device, thecomputer system may still select the content for insertion even when thefirst context is not detected.

FIG. 5 illustrates an example of a network environment for providingcontent based on contexts, according to an embodiment of the presentdisclosure. As illustrated, the network environment may include acomputer system 500 and a web server 550. The web server 550 may host aweb site and may transmit a content request 552 to the computer system,where this content request 552 is associated with a web page of the website. In response, the computer system 500 may identify content 502 tothe web server 550 and may instruct the web server 550 to insert thecontent 502 in the web page. These interactions between the computersystem 500 and the web server 550 may rely on contexts andcontext-content associations, some of which may have been generated bythe computer system 500 as described herein above in connection withFIGS. 3 4. In such cases, the interactions may represent examples ofintent satisfaction.

As illustrated, the computer system 500 may include multiple computingresources, such as a context augmentation server 510, a data store 520,and a content server 530. Some or all of these computing resources maybe similar to the ones of computer system 200 of FIG. 2. A serviceprovider may manage these computing resources for the intentsatisfaction. In an example, the web server 550 may, but need not be,also managed by the service provider. Interactions between the computingresources are further described in the next figure.

The context augmentation server 510 may receive the context request 552from the web server 550. In response, the context augmentation server510 may augment the content request 552 by identifying one or morecontexts relevant to the content request 552 and adding the context(s)to the content request 552. Some of the identified contexts may bepre-computed (e.g., generated and stored prior to receiving the contentrequest 552). For instance, contexts associated with web pages such askeywords found in the web pages and attributes of the keywords may bepre-computed and already available as pre-computed contexts 522 from thedata store 520. Other identified contexts may be computed on-the-fly(e.g., in near real-time upon receiving the content request 552).Contexts associated with a visitor and a visitor device may be computedon-the-fly based on an identifier of the visitor, an identifier of thevisitor device, a location of the visitor device, the timing of thecontent request 552, and/or profile information about the visitor and/orvisitor device.

In an example, the pre-computed contexts 522 may represent contextsderived by the service provider by executing the various workflows usingdifferent intent descriptions. For instance, each time an intentdescription of a content provider is received and the resultingcontext-content associations are identified, the service provider mayalso store the identified contexts as pre-computed contexts. Similarly,the content provider may input its own intent descriptions, generatecontext-content associations (without necessarily storing these), andstore the identified contexts as also pre-computed contexts.

The augmented content request may be sent to the content server 530. Inresponse, the content server 530 may select and identify the content 502to the web server 550. To do so, the content server 530 may storecontext-content associations 532 in its cache. These context-contentassociations 532 may be retrieved periodically from context-contentassociations 524 that are already stored in the data store 520. Thecontext-content associations 524 of the data store 520 may have beengenerated based on intent descriptions of content providers asillustrated herein above in connection with FIGS. 3-4. The retrieval mayinclude a periodic pull or a push upon an addition of newcontext-content associations. By caching the context-contentassociations 532, the content server 530 may more quickly identify thecontent 502. Nonetheless, if no match is detected from the cache, thecontent server 530 may asynchronously query the data store 520 (or aquery engine thereof) for candidate matches from the context-contentassociations 524.

The web server 550 may also store the different content 534 of thedifferent content providers. This storing may include storing the actualcontent or identifiers of the actual content (e.g., URLs of the networklocations where the content 534 may be stored).

The content server 530 may also host a content manager 536 to select thecontent 502 from the content 534 based on the context-contentassociations 532 (or the context-content associations 524 if there wasno match in the cache). In an example, the content manager 536 mayinclude a matcher 537 and an AI model 539.

The matcher 537 may be configured to match the context(s) from theaugmented content request with context(s) from the context-contentassociations 532 (or the context-content associations 524 if there wasno match in the cache). The matching may be a Boolean matching. Forinstance, one particular content may be associated with a set ofpre-computed contexts including a first pre-computed context and asecond pre-computed context. In comparison, the augmented contentrequest may include a set of added contexts including a first contextand a second context. If the added contexts match the pre-computedcontexts (e.g., the first context matches the first pre-computedcontext, and the second context matches the second pre-computedcontext), the matcher 537 may identify a match between the contentrequest 552 and the particular content and this content may becomecandidate content for selection and insertion. However, if a mismatchexists (e.g., either the first context or the second context does notmatch the first pre-computed context or the second pre-computed context,respectively), the matcher 537 may identify a mismatch between thecontent request 552 and the particular content and the particularcontent may not be added (or may be removed) from the set of candidatecontent.

The AI model 539 may be configured to generate a relevancy score forcontent of a content provider based on historical behavior data, theapplicable intent description of the content provider, the matchedpre-computed contexts, and the content. For example, the AI model 539may implement a ML algorithm. This algorithm may be trained withhistorical behavior data 526 stored at the data store. The historicalbehavior data 526 may include historical intent descriptions, contexts,content, and outcomes of presenting the content given the intentdescriptions and contexts. The outcomes may be used as ground truth totrain the ML algorithm to generate relevancy scores. The training mayminimize a loss of the ML algorithm by tuning parameters of the MLalgorithm to best approximate the relevancy scores based on inputs thatinclude the historical intent descriptions, contexts, and content andknowing the historical outcomes (e.g., the loss function is reduced whenthe ML algorithm outputs a high relevancy score for an input thatincludes an intent description having a conversion goal and the outcomeindicates a conversion; otherwise, the loss function is increased). Inthis way, the ML algorithm may learn or be trained to recognize featuresinherent to intent descriptions, contexts, and content.

Once trained, the AI model 539 may receive, for each candidate contentof a content provider, the pre-computed contexts that were matched, thecorresponding intent description of the content provider, and thecandidate content. This information may be provided as an input the AImodel 539 or a feature map may be generated from the information andprovided as the input. In response, the AI model 539 may output arelevancy score the candidate content. Accordingly, the differentcandidate content may be ranked and the content server 530 may selectthe content 502 from the candidate content based on the relevancy scoresand other factors (e.g., including placement rate, visitor fatigue, andthe like).

FIG. 6 illustrates an example of calls between computing resources forproviding content based on contexts, according to an embodiment of thepresent disclosure. In an example, the interactions may include calls,such as API calls to exchange data, execute processes, and generatecontexts. Some or all of the computing resources may be managed by aservice provider and may be the same as those described herein above inconnection with FIG. 5. In particular, the computing resources mayinclude a context augmentation server 620, a data store 630, and acontent server 640.

As illustrated, a web server 610 may send a content request 612 to thecontext augmentation server 620 based on a visitor device's accessrequest to a web page. The content request 612 may identify the web pageand may include a pre-computed context associated with the web page(e.g., a keyword).

In response, the context augmentation server 620 may identify additionalcontexts to add to the content request 612. Some of the additionalcontexts, such as ones associated with the web page, may bepre-computed, while other additional contexts, such as about the visitoror the visitor device (e.g., device location, timing, etc.) may becomputed on-the-fly. To identify the pre-computed contexts, the contextaugmentation server 620 may generate and transmit a context query 622 tothe data store 630. The context query 622 may identify the web page andany context already included in the content request 612.

A query engine of the data store 630 may return pre-computed contexts632 to the content augmentation server 620. In an example, the queryengine may identify the various pre-computed contexts 632 that arealready associated with the web page. For instance, the data store 630may store various pre-computed contexts and associations of thesecontexts with web pages. In this example, the query engine may look upthe stored pre-computed contexts 632 using the identifier of the webpage from the context query 622 to then return the set of pre-computedcontexts 632 associated with the web page. In another example, the queryengine may identify the various pre-computed contexts that are alreadyassociated with the context(s) included in the context query 622. Inparticular, and similarly to storing associations of pre-computedcontexts with web pages, the data store 630 may store associationsbetween the pre-computed contexts themselves. Hence, the query enginemay look up these pre-computed contexts 632 associations using thecontexts(s) from the context query 622 to then return the set ofassociated pre-computed contexts 632. In yet another example, the queryengine may identify a first pre-computed context associated with the webpage or the context from the content request 622 as described in the twoprevious examples. The query engine may also identify a secondpre-computed context as an expected content given the first pre-computedcontext. For instance, the query engine may input the first pre-computedcontext to a prediction engine that then output the second pre-computedcontext as described herein above. The second pre-computed context maybe returned to the context augmentation server 620. Hence, thepre-computed contexts 632 may include a pre-computed context(s)determined based on the identifier of the web page, a pre-computedcontext(s) identified based on any context from the context request 612,and/or a pre-computed context(s) identified based as an expected contextgiven another identified context.

The context augmentation server 620 may update the content request 612to include the pre-computed contexts 632, thereby generating anaugmented context request 624. The augmented context request 624 may betransmitted to the content server 640.

The content server 640 may select content 642 based on the augmentedcontext request 624. In an example, the selection may include contextmatching to identify a set of candidate content and content scoring toselect the content 642 from the set. For the content matching, thecontent server 640 may match the pre-computed contexts 632 included inthe augmented context request 624 with contexts from context-contentassociations available from the data store. Particular content is addedto the set of candidate content (or removed from the set) based on amatch between the pre-computed contexts from the augmented contextrequest 624 and the contexts from the context-content associations ofthe particular content. For the content scoring, a relevancy score maybe generated for each of the candidate content in the set. The relevancyscore for particular content of a content provider may be generated byan AI model based on an input thereto. The input may include the matchedpre-computed contexts, the relevant intent description of the contentprovider, and/or the particular content. The relevancy score may be usedas one of the factors for the selection of the content 624 from the set.The content server 640 may send the content 642 to the web server forinsertion in the web page.

FIG. 7 illustrates an example flow for providing content based oncontexts, according to an embodiment of the present disclosure.Instructions for performing the operations of the flow can be stored ascomputer-readable instructions on a non-transitory computer-readablemedium of a computer system, such as the computer systems describedherein above in connection with FIGS. 1-2 and 5. As stored, theinstructions represent programmable modules that include code or dataexecutable by a processor(s) of the computer system. The execution ofsuch instructions configures the computer system to perform the specificoperations shown in the figures and described herein. Each programmablemodule in combination with the processor(s) represents a means forperforming a respective operation(s). While the operations areillustrated in a particular order, it should be understood that noparticular order is necessary and that one or more operations may beomitted, skipped, and/or reordered.

As illustrated, the example flow may start at operation 702, where thecomputer system may receive a request for content. In an example, a webserver may send the content request to the computer system based on anaccess request of a visitor device to a web page. The content requestmay identify the web page and, include one or more contexts.

At operation 704, the computer system may determine contexts associatedwith the request. In an example, different context types may beidentified. For instance, a first context may be associated with the webpage, a second context may be associated with a context from the contentrequest, and a third context may be associated with context identifiedbased on the web page or the context from the content request. Suchcontexts may be pre-computed and identifying them may rely on storedassociations between web pages and pre-computed contexts or betweenpre-computed contexts themselves. Other context types may be alsoidentified, such as a context associated with a visitor that isoperating the visitor device and/or with the visitor device.

At operation 706, the computer system may update the content request toadd the context(s) determined at operation 704. In an example, thecomputer system may generate an augmented content request by adding thedetermined context(s).

At operation 708, the computer system may determine a match between thecontext(s) from the updated content request (e.g., the augmented contentrequest) and content of a content provider. In an example, theparticular content may be pre-associated with contexts and theseassociations may be stored as context-content associations. The computersystem may determine a match between all of the context(s) from theupdated content request and contexts from the context-contentassociations of the particular content. Based on the match, theparticular content may be identified as candidate content. If a mismatchexist, the particular content may not be candidate content.

At operation 710, the computer system may generate a set of candidatecontent. For example, the computer system may match the pre-computedcontexts from the updated content request with multiple content, underoperation 708. Each time a match is determined, the correspondingcontent may be added to the set of candidate content.

At operation 712, the computer system may generate a score for eachcandidate content in the set of candidate content. In an example, eachcandidate content corresponds to an intent description of a contentprovider. That intent description, along other information (e.g., thematched pre-computed contexts and/or the particular candidate contentitself) may be input to an AI model hosted by the computer system. TheAI model may be trained to generate the relevancy score based onhistorical behavior data, intent descriptions, contexts, and content.

At operation 714, the computer system may select content from the set ofcandidate content based on the score of this content and relevancyscores of other candidate content, among other factors. In an example,the computer system may rank the candidate content from the set based ontheir scores and may use this ranking along with, for instance,placement rate and visitor fatigue, to select the content.

At operation 716, the computer system may provide the content forpresentation at the user device. In an example, the computer system maysend, to the web server in response to the content request, anidentifier of the content (e.g., a URL of a content source storing thecontent) along with instructions to insert the content in the web page.Accordingly, when the web page is presented at the visitor device, thepresented web page may include the content.

FIG. 8 illustrates a computer architecture diagram showing an examplecomputer architecture, according to an embodiment of the presentdisclosure. This architecture may be used to implement some or all ofthe systems described herein. The computer architecture shown in FIG. 8illustrates a conventional server computer, workstation, desktopcomputer, laptop, tablet, network appliance, personal digital assistant(“PDA”), e-reader, digital cellular phone, or other computing device,and may be utilized to execute any aspects of the software componentspresented herein.

The computer 800 includes a baseboard 802, or “motherboard,” which is aprinted circuit board to which a multitude of components or devices maybe connected by way of a system bus or other electrical communicationpaths. In one illustrative embodiment, one or more central processingunits (“CPUs”) 804 operate in conjunction with a chipset 806. The CPUs804 may be standard programmable processors that perform arithmetic andlogical operations necessary for the operation of the computer 800.

The CPUs 804 perform operations by transitioning from one discrete,physical state to the next through the manipulation of switchingelements that differentiate between and change these states. Switchingelements may generally include electronic circuits that maintain one oftwo binary states, such as flip-flops, and electronic circuits thatprovide an output state based on the logical combination of the statesof one or more other switching elements, such as logic gates. Thesebasic switching elements may be combined to create more complex logiccircuits, including registers, adders-subtractors, arithmetic logicunits, floating-point units, and the like.

The chipset 806 provides an interface between the CPUs 804 and theremainder of the components and devices on the baseboard 802. Thechipset 806 may provide an interface to a random access memory (“RAM”)808, used as the main memory in the computer 800. The chipset 806 mayfurther provide an interface to a computer-readable storage medium suchas a read-only memory (“ROM”) 810 or non-volatile RAM (“NVRAM”) forstoring basic routines that help to startup the computer 800 and totransfer information between the various components and devices. The ROM810 or NVRAM may also store other software components necessary for theoperation of the computer 800 in accordance with the embodimentsdescribed herein.

The computer 800 may operate in a networked environment using logicalconnections to remote computing devices and computer systems through anetwork, such as the local area network 820. The chipset 806 may includefunctionality for providing network connectivity through a NIC 812, suchas a gigabit Ethernet adapter. The NIC 812 is capable of connecting thecomputer 800 to other computing devices over the network 820. It shouldbe appreciated that multiple NICs 812 may be present in the computer800, connecting the computer to other types of networks and remotecomputer systems.

The computer 800 may be connected to a mass storage device 818 thatprovides non-volatile storage for the computer. The mass storage device818 may store system programs, application programs, other programmodules, and data, which have been described in greater detail herein.The mass storage device 818 may be connected to the computer 800 througha storage controller 814 connected to the chipset 806. The mass storagedevice 818 may consist of one or more physical storage units. Thestorage controller 814 may interface with the physical storage unitsthrough a serial attached SCSI (“SAS”) interface, a serial advancedtechnology attachment (“SATA”) interface, a fiber channel (“FC”)interface, or other type of interface for physically connecting andtransferring data between computers and physical storage units.

The computer 800 may store data on the mass storage device 818 bytransforming the physical state of the physical storage units to reflectthe information being stored. The specific transformation of physicalstate may depend on various factors, in different implementations ofthis description. Examples of such factors may include, but are notlimited to, the technology used to implement the physical storage units,whether the mass storage device 818 is characterized as primary orsecondary storage, and the like.

For example, the computer 800 may store information to the mass storagedevice 818 by issuing instructions through the storage controller 814 toalter the magnetic characteristics of a particular location within amagnetic disk drive unit, the reflective or refractive characteristicsof a particular location in an optical storage unit, or the electricalcharacteristics of a particular capacitor, transistor, or other discretecomponent in a solid-state storage unit. Other transformations ofphysical media are possible without departing from the scope and spiritof the present description, with the foregoing examples provided only tofacilitate this description. The computer 800 may further readinformation from the mass storage device 818 by detecting the physicalstates or characteristics of one or more particular locations within thephysical storage units.

In addition to the mass storage device 818 described above, the computer800 may have access to other computer-readable storage media to storeand retrieve information, such as program modules, data structures, orother data. It should be appreciated by those skilled in the art thatcomputer-readable storage media can be any available media that providesfor the storage of non-transitory data and that may be accessed by thecomputer 800.

By way of example, and not limitation, computer-readable storage mediamay include volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology. Computer-readable storage mediaincludes, but is not limited to, RAM, ROM, erasable programmable ROM(“EPROM”), electrically-erasable programmable ROM (“EEPROM”), flashmemory or other solid-state memory technology, compact disc ROM(“CD-ROM”), digital versatile disk (“DVD”), high definition DVD(“HD-DVD”), BLU-RAY, or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other medium that can be used to store the desired information ina non-transitory fashion.

The mass storage device 818 may store an operating system 830 utilizedto control the operation of the computer 800. According to oneembodiment, the operating system comprises the LINUX operating system.According to another embodiment, the operating system comprises theWINDOWS® SERVER operating system from MICROSOFT Corporation. Accordingto further embodiments, the operating system may comprise the UNIX orSOLARIS operating systems. It should be appreciated that other operatingsystems may also be utilized. The mass storage device 818 may storeother system or application programs and data utilized by the computer800. The mass storage device 818 might also store other programs anddata not specifically identified herein.

In one embodiment, the mass storage device 818 or othercomputer-readable storage media is encoded with computer-executableinstructions which, when loaded into the computer 800, transforms thecomputer from a general-purpose computing system into a special-purposecomputer capable of implementing the embodiments described herein. Thesecomputer-executable instructions transform the computer 800 byspecifying how the CPUs 804 transition between states, as describedabove. According to one embodiment, the computer 800 has access tocomputer-readable storage media storing computer-executable instructionswhich, when executed by the computer 800, perform the various routinesdescribed above. The computer 800 might also include computer-readablestorage media for performing any of the other computer-implementedoperations described herein.

The computer 800 may also include one or more input/output controllers816 for receiving and processing input from a number of input devices,such as a keyboard, a mouse, a touchpad, a touch screen, an electronicstylus, or other type of input device. Similarly, the input/outputcontroller 816 may provide output to a display, such as a computermonitor, a flat-panel display, a digital projector, a printer, aplotter, or other type of output device. It will be appreciated that thecomputer 800 may not include all of the components shown in FIG. 8, mayinclude other components that are not explicitly shown in FIG. 8, or mayutilize an architecture completely different than that shown in FIG. 8.It should also be appreciated that many computers, such as the computer800, might be utilized in combination to embody aspects of the varioustechnologies disclosed herein.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the disclosure asset forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit theinvention to the specific form or forms disclosed, but on the contrary,the intention is to cover all modifications, alternative constructionsand equivalents falling within the spirit and scope of the invention, asdefined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including,”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected” is to be construed as partly or wholly contained within,attached to, or joined together, even if there is something intervening.Recitation of ranges of values herein are merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range, unless otherwise indicated herein, and eachseparate value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (e.g., “such as”) provided herein, isintended merely to better illuminate embodiments of the invention anddoes not pose a limitation on the scope of the invention unlessotherwise claimed. No language in the specification should be construedas indicating any non-claimed element as essential to the practice ofthe invention.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the invention.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Theinventors expect skilled artisans to employ such variations asappropriate and the inventors intend for the invention to be practicedotherwise than as specifically described herein. Accordingly, thisinvention includes all modifications and equivalents of the subjectmatter recited in the claims appended hereto as permitted by applicablelaw. Moreover, any combination of the above-described elements in allpossible variations thereof is encompassed by the invention unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications and patents,cited herein are hereby incorporated by reference to the same extent asif each reference were individually and specifically indicated to beincorporated by reference and were set forth in its entirety herein.

What is claimed is:
 1. A method implemented on a computer system,comprising: providing a graphical user interface to a computing deviceof a content provider, the graphical user interface comprising an inputfield configured to receive natural language input; receiving, from thecomputing device, an intent description based at least in part on theinput field of the graphical user interface, the intent descriptionassociated with a presentation of content of the content provider at aweb site; determining a goal and a predicate for a query data structurebased at least in part on natural language processing of the intentdescription; sending a query to a data store storing a plurality ofworkflows, the query having the query data structure and comprising thegoal and the predicate; receiving, from the data store and based atleast in part on the query, an identifier of a workflow from theplurality of workflows, the workflow comprising executable instructionsassociated with determining a context of a web page of the web sitebased at least in part on historical behavior data of web visitors tothe web site and on a taxonomy of the web site; determining the contextof the web page based at least in part on an execution of the workflow,the historical behavior data, and the taxonomy of the web site, thecontext specific to the web page and comprising a plurality of keywordsand an attribute of the plurality of keywords; storing, in the datastore, an association between the context and the content; andinstructing a web server to insert the content in the web page based atleast in part on a detection of the context and on the association inthe data store.
 2. The method of claim 1, further comprising: sending,to the graphical user interface, the context and an option to select thecontext from a plurality of contexts; and generating the associationbased at least in part on a selection of the option at the graphicaluser interface.
 3. The method of claim 1, further comprising:determining, based at least in part on the intent description, candidateweb pages of the web site; identifying candidate keywords from thecandidate web pages, the keywords corresponding to items described inthe candidate web pages; and computing a similarity measurement betweenthe candidate keywords, wherein the workflow comprises a keywordidentification workflow that is selected based at least in part on thesimilarity measurement indicating a dissimilarity between attributes ofthe candidate keywords, and wherein the context comprises a keywordbased at least in part on an execution of the keyword identificationworkflow.
 4. The method of claim 1, further comprising: determining,based at least in part on the intent description, candidate web pages ofthe web site; identifying candidate keywords from the candidate webpages, the keywords corresponding to items described in the candidateweb pages; and computing a similarity measurement between the candidatekeywords, wherein the workflow comprises an attribute identificationworkflow that is selected based at least in part on the similaritymeasurement indicating a similarity between attributes of the candidatekeywords, and wherein the context comprises the attribute of theplurality of keywords based at least in part on an execution of theattribute identification workflow.
 5. A computer system, comprising: aprocessor; and a memory storing computer-readable instructions that,upon execution by the processor, configure the computer system to:receive, from a computing device of a content provider, an intentdescription associated with providing content at a network resource thatcomprises a plurality of network documents; determine a goal and apredicate for a query data structure based at least in part on naturallanguage processing of the intent description; send a query to a datastore storing a plurality of workflows, the query having the query datastructure and comprising the goal and the predicate; receive, from thedata store and based at least in part on the query, an identifier of aworkflow from the plurality of workflows, the workflow comprisingexecutable instructions associated with determining a context of anetwork document based at least in part on historical behavior data ofvisitors to the network resource and on a taxonomy of the networkresource; determine the context based at least in part on an executionof the workflow, the historical behavior data, and the taxonomy, thecontext specific to the network document and comprising a plurality ofkeywords and an attribute of the plurality of keywords; and store anassociation between the context and the content, the association beingusable to select the content for presentation in the network document.6. The computer system of claim 5, wherein the computer-readableinstructions further configure the computer system to: provide agraphical user interface to the computing device of the contentprovider, wherein the graphical user interface comprises an input fieldconfigured to receive natural language input, wherein the intentdescription is received based at least in part on the input field,wherein the network resource comprises a web site, wherein the networkdocument comprises a web page of the web site, wherein the associationis stored in the data store; and instruct a web server to insert thecontent in the web page based at least in part on a detection of thecontext and on the association in the data store.
 7. The computer systemof claim 5, wherein the goal is determined based at least in part on aquery understanding of the intent description, and wherein the predicateis determined based at least in part on the query understanding and thetaxonomy of the network resource.
 8. The computer system of claim 5,wherein the intent description is received based at least in part oninput from the computing device, wherein the input identifies thenetwork resource, and wherein the computer-readable instructions furtherconfigure the computer system to: determine a second context of a secondnetwork document from a second network resource, the second contextdetermined based at least in part on the workflow, second historicalbehavior data of second visitors to the second network resource, and asecond taxonomy of the second network resource; determine, based atleast in part on the second context and the second historical behaviordata, a performance measurement associated with presenting the contentat the second network document; and provide, to the computing device, arecommendation to associate the content with the second networkresource.
 9. The computer system of claim 5, wherein the computer systemis associated with a service provider, and wherein the computer-readableinstructions further configure the computer system to provide aninterface to a computing device of the service provider, wherein theinterface comprises workflow fields, and wherein the workflow isgenerated and stored in the data store based at least in part on inputof the service provider at the workflow fields of the interface.
 10. Thecomputer system of claim 5, wherein the association is stored in a datastore as a key-value pair, wherein a key of the key-value pair comprisesthe context, and wherein a value of the key-value pair comprises anidentifier of the content.
 11. The computer system of claim 10, whereinthe computer-readable instructions further configure the computer systemto: store the key-value pair in cache, and wherein the content isinserted in the network document based at least in part on a detectionof the context and on the key-value pair in the cache.
 12. The computersystem of claim 11, wherein the computer-readable instructions furtherconfigure the computer system to: generate a second key-value pair basedat least in part on the association between the context and the content,wherein a key of the second key-value pair comprises the identifier ofthe content, and wherein a value of the second key-value pair comprisesthe context; and in response to a query from the computing device of thecontent provider about the content, provide the context to the computingdevice based at least in part on the second key-value pair.
 13. Acomputer-readable storage medium comprising instructions that, uponexecution on a computer system, configure the computer system to performoperations comprising: receiving, from a computing device of a contentprovider, an intent description associated with providing content at anetwork resource that comprises a plurality of network documents;determining a goal and a predicate for a query data structure based atleast in part on natural language processing of the intent description;sending a query to a data store storing a plurality of workflows, thequery having the query data structure and comprising the goal and thepredicate; receiving, from the data store and based at least in part onthe query, an identifier of a workflow from the plurality of workflows,the workflow comprising executable instructions associated withdetermining a context of a network document based at least in part onhistorical behavior data of visitors to the network resource and on ataxonomy of the network resource; determining the context based at leastin part on an execution of the workflow, the historical behavior data,and the taxonomy the context specific to the network document andcomprising a plurality of keywords and an attribute of the plurality ofkeywords; and storing an association between the context and thecontent, the association being usable to select the content forpresentation in the network document.
 14. The computer-readable storagemedium of claim 13, wherein the intent description is received from aninterface presented at the computing device of the content provider, andwherein the operations further comprise: determining, based at least inpart on the context and the historical behavior data, a performancemeasurement associated with presenting the content at the networkdocument; and providing the performance measurement to the computingdevice, wherein the performance measurement is presented at theinterface in association with the intent description.
 15. Thecomputer-readable storage medium of claim 14, wherein the operationsfurther comprise: receiving, based at least in part on a presentation ofthe performance measurement at the interface, an updated intentdescription from the computing device of the content provider; selectinga second workflow from the plurality of workflows based at least in parton the updated intent description; and determining a second context ofthe network document based at least in part on an execution of thesecond workflow, the historical behavior data, and the taxonomy.
 16. Thecomputer-readable storage medium of claim 14, wherein the interfacecomprises a performance field that presents the performance measurement,a plurality of context fields that present a plurality of determinedcontexts, and option fields available to select one or more of theplurality of determined contexts, and wherein the operations furthercomprise: receiving, from the computing device of the content provider,a selection of a subset of contexts from the plurality of determinedcontexts based at least in part on the option fields; determining, basedat least in part on the subset, an updated performance measurement; andproviding the updated performance measurement to the computing device,wherein the updated performance measurement is presented in theperformance field of the interface.
 17. The computer-readable storagemedium of claim 13, wherein the operations further comprise:determining, based at least in part on the context and the historicalbehavior data, a second context of a second network document, whereindetection of the second context is expected based at least in part on avisitor accessing the network document; and storing a second associationbetween the second context and second content of a second contentprovider.
 18. The computer-readable storage medium of claim 13, whereinthe intent description is received based at least in part on input fromthe computing device at an interface, wherein the interface identifies aplurality of network resource, and wherein the input selects the networkresource from the plurality of network resources.
 19. Thecomputer-readable storage medium of claim 13, wherein the workflow isexecuted at a first point in time to determine the context, and whereinthe operations further comprise: determining a second context based atleast in part on a re-execution of the workflow at a second point intime, wherein the workflow is automatically re-executed at the secondpoint based at least in part on updated historical behavior data andwhile the intent description remains the same.
 20. The computer-readablestorage medium of claim 13, wherein the operations further comprise:determining a second context by at least re-executing the workflow basedat least in part on a change to the taxonomy of the network resource.